Skip to content

i18n: translate the 18 never-translated curated pages, and re-sync 12 drifted pages - #1964

Merged
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/backfill-and-resync-2026-08-15
Aug 15, 2026
Merged

i18n: translate the 18 never-translated curated pages, and re-sync 12 drifted pages#1964
zksquirrel merged 3 commits into
ZecHub:mainfrom
bloxster:i18n/backfill-and-resync-2026-08-15

Conversation

@bloxster

Copy link
Copy Markdown
Contributor

What this does

Two things, and the corpus ends up fully current in all 18 locales for the first time.

1. Backfill — 18 curated pages that had never been translated in any language. These were declared as needing translation but never produced: 16 came from #1944 ("Curate the 22 Ironwood-era pages missing from the translation list") and Sapling/Sprout returned via #1946. git log --all --diff-filter=A finds no commit that ever added a translation for any of them, so this is a gap that has existed since the day they were curated, not drift.

Mostly the protocol timeline: Overwinter, Sapling, Blossom, Heartwood, Canopy, NU5, NU6, NU6_1, NU6_2, Ironwood, plus Zakura_Node, ZECD, What_a_Block_Explorer_Can_See, Zecmap, Zallet_Quick_Reference_Guide, Zcash_Engineering_Office_Hours and ZecHub_Hackathon_Builders_Guide.

18 pages × 18 locales = 324 items, 0 failures, 0 held.

2. Re-sync — 12 pages that drifted since the last sync. Three English PRs merged in the meantime (#1948's Tor/I2P rewrite, #1957's Zashi→Zodl rename, and others), leaving 205 stale locale-pages. All re-synced: 205 items, 0 failures, 0 held.

Result

check before after
detect-staleness — missing 324 0
detect-staleness — stale 205 7 (deliberate, below)
check-protected-terms (whole tree) 72 violations passes for all 3,654 translated pages
check-invariants --base holds, 203 pages × 18 locales
menu-titles --check clean
hash-lib tests 38 pass / 0 fail

3,654 = 203 curated pages × 18 locales. Every curated page now exists in every locale and carries every protected term.

The 7 deliberate stale entries

Using_Zcash/Zcash_Mining_Pools.md in de/hi/it/ja/ru/tr/uk keep their base provenance. Its English delta touched only non-translatable content (the /content-images/*.webp URL migration), so diff-aware re-translation reproduces byte-identical output and the agent restamps src alone — which trips change-tracking with "manifest src changed but the translation file did not" (check-invariants.mjs:263), a direction with no escape hatch. The translations are already correct; 9aea0b1b propagated those URLs.

Keeping base provenance understates freshness rather than overstating it. The cost is a loop: they read as stale, the next run re-attempts them, produces byte-identical output, restamps, and fails the gate again — that cycle needed two manual interventions today alone. The durable fix is a verifiable escape hatch comparing the translatable-text projection of English at the two commits and allowing the bump when unchanged; a self-declared manifest flag would not be checkable and shouldn't be used.

One thing worth knowing about how this was verified

The backfill initially failed the scoped gate with 18 introduced violations — all Zingo, on ZECD.md, one per locale. Rather than patch it, I traced it: the base English had 0 occurrences of Zingo, main's has 1, introduced by a9595ae1 (#1957). The backfill had translated that page correctly against the English it had; main moved underneath it. The re-sync commit fixes it properly.

That is the scoped gate from #1958 behaving exactly as intended — a newly added translation missing a required term is precisely what "introduced" should catch, and it caught it before this PR was opened.

Review notes

Machine-translated content is mechanics-reviewed, not linguistically reviewed — nobody can meaning-review 18 languages and this PR doesn't pretend otherwise. What is verified is structure, protected terminology, manifest coherence, and that diff-aware paths changed only the blocks they should. Native-speaker corrections are welcome afterwards; if you hand-fix a translation, record it in tool on that manifest entry, and use edited: true only for a page you intend to maintain by hand — it removes the page from automated sync permanently.

Engine split as usual: 13 locales via LLM (diff-aware where possible), sw/yo/ig via NLLB, ak/ee via GT-static.

…Pools entries

Second occurrence today, and the loop is now demonstrated rather than
predicted. The English delta on Using_Zcash/Zcash_Mining_Pools.md touched
only non-translatable content (the /content-images/*.webp URL migration), so
diff-aware re-translation reproduces byte-identical output and the agent
restamps src alone. check-invariants' change-tracking then fires: 'manifest
src changed but the translation file did not' (check-invariants.mjs:263),
which has no escape hatch.

Reverting the provenance makes those entries read as stale, so the NEXT run
picks them up, produces byte-identical output again, restamps again, and
fails the gate again. That cycle burned two manual interventions today and
will burn one per sync until it is fixed properly.

The durable fix is a verifiable escape hatch: compare the translatable-text
projection of English at the two commits and permit the bump when it is
unchanged. The projection must be conservative in the opposite direction
from normalize-hash — when in doubt REFUSE the bump, since wrongly allowing
one marks a stale translation fresh. Tracked as a follow-up.
@zksquirrel
zksquirrel merged commit 636861a into ZecHub:main Aug 15, 2026
4 checks passed
zksquirrel pushed a commit that referenced this pull request Aug 17, 2026
…passed 1 MiB

`check-invariants.mjs` read the base manifest with `execFileSync` and no
`maxBuffer`, so it inherited Node's 1 MiB default. `translation/sync-state.json`
crossed that threshold at 2dd8fb0 (the 18-page sync merged in #1964, 1,001,048
-> 1,094,558 bytes), and from that commit on `git show` threw ENOBUFS on every
run.

The read sat in a catch that treated any failure as "the manifest did not exist
at the base — first introduction, nothing to change-track". So both
change-tracking directions were skipped on every pull request since 2026-08-15
while the job printed `Manifest invariants hold` and stayed green. The bijection
checks don't touch that read, which is precisely why nothing looked wrong.

What was unenforced in that window:

- direction 1 — a translated file could change with no provenance trace, the
  invariant that stops a hand-edit from being indistinguishable from a page
  nobody re-synced;
- direction 2 — `src` could be bumped without the file changing, marking a stale
  translation fresh forever. That is the single lie the manifest exists to
  prevent, and by design it has no escape hatch.

The fix, and two adjacent instances of the same mistake found while reviewing it:

- Bound every git read in this file at 64 MiB, matching what
  scripts/check-protected-terms.mjs already does. `git ls-files translations/`
  is 210 KB today and scales with locales x pages, so it is on the same path.

- Establish absence instead of inferring it, using `git ls-tree` rather than
  `git cat-file -e`. Absent and present are both exit 0 under `ls-tree` and are
  told apart by the output, so a non-zero exit is unambiguously an error.
  `cat-file -e` cannot draw that line: it exits non-zero both for an absent path
  and for a blob it could not obtain, so under `--filter=blob:none` an
  unreachable promisor remote would read as "absent" and skip the gate again —
  the same silent skip with a network trigger. `ls-tree` also never needs the
  blob at all.

- Reject a base manifest that parses but is not an object. `null`, an array or a
  string would parse and then throw on the first `baseManifest[loc]`.

- Fail on `--base` with no ref value. It resolved to `undefined`, which read as
  "no base requested" and skipped change-tracking with a notice and exit 0. CI
  invokes `--base "$BASE_REF_OUT"`, so an empty variable would have landed
  exactly there. Asking for the gate and silently not getting it is the failure
  this whole block exists to prevent.

Verified on this tree, base origin/main:

| case | before | after |
|---|---|---|
| clean tree | "no manifest", skipped | change-tracked, holds |
| translation edited, manifest untouched | passed green | fails, direction 1 |
| `src` bumped, file untouched | passed green | fails, direction 2 |
| manifest genuinely absent at base | notice, skip | notice, skip (unchanged) |
| base manifest is `null` | TypeError stack trace | fails, names the reason |
| `--base` with no value | notice, exit 0 | fails, exit 1 |
| no `--base` at all | notice, exit 0 | notice, exit 0 (unchanged) |

Reviewed by two independent adversarial passes. One reproduced the mask on a
real violation and established the `cat-file -e` / `ls-tree` distinction
empirically in a synthetic blobless clone: with the promisor unreachable,
`cat-file -e` exits 128 for a blob that exists, indistinguishable from a genuine
absence, while `ls-tree` answers correctly from the tree objects. The other found
the `--base` and base-shape defects. Both are fixed here.

Not fixed, filed instead: unbounded git reads in detect-staleness.mjs,
seed-sync-state.mjs and gen-menu-titles.mjs (all 30-2000x under the limit and
scaling linearly rather than with the locale x page product), and this file's
`ls-files` calls not using `-z`, which seed-sync-state.mjs documents as a
non-ASCII filename trap.
zksquirrel pushed a commit that referenced this pull request Aug 19, 2026
`pages build and deployment` has been failing since 2026-08-14, and the published
GitHub Pages site (https://zechub.github.io/zechub/, status `errored`) has been
frozen on that day's build ever since.

The LLM sync prepends a `---` that the English source does not have. Jekyll reads
a leading `---` as a YAML front-matter opener, runs to the next `---` far down the
page, and fails to parse. The build dies on
translations/ja/site/Zcash_Tech/What_a_Block_Explorer_Can_See.md.

Emulating Jekyll's front-matter parse across the whole tree: 251 translated pages
break, 0 English pages do. Every breaking file is a translation, and in all 251
the English source has no leading `---`, so there is no judgment call — the
dashes are simply not supposed to be there.

Traced by counting leading-`---` files at each merge:

| commit | files | pages build |
|---|---|---|
| d951213 (#1948) | 96 | last success |
| c301ff2 (#1960) | 127 | first failure |
| 636861a (#1964) | 356 | failing |

96 such files predate this and the build was green, so a leading `---` is not
automatically fatal — it breaks when the resulting pseudo-front-matter is invalid
YAML, which prose in these locales reliably produces. #1960 tipped it over and
#1964 made it much worse.

This strips the stray line from all 260 pages where the English source lacks it,
not only the 251 that currently break, since the other 9 are the same defect and
a latent break.

Reader-facing impact was limited: the frontend calls next-mdx-remote's
`serialize(..., {})`, whose `parseFrontmatter` defaults to false, and gray-matter
— though in package.json — is imported nowhere in src/, so the dashes rendered as
a stray horizontal rule rather than swallowing content. Worth noting gray-matter
DOES throw on these files, so anything that starts using it would break hard.

Two sidebar labels were wrong, though, and this fixes them: `Using_Zcash/Zecmap.md`
read "Kurzfassung" in de and "Resumo" in pt — both meaning "summary" — because the
stray `---` made the title extractor skip `# ZecMap` and pick up `## TL;DR`. Both
now read "ZecMap". Menu-title manifests regenerated accordingly.

No prose changed: the only content line touched anywhere is the bare `---`.
`src` is untouched because it hashes the ENGLISH source, which has not moved.
`edited` stays false throughout — this is a mechanical pass, and flipping it
would remove these pages from automated sync permanently.

Verification, all on this branch:

| check | result |
|---|---|
| Jekyll front-matter emulation | 0 breaking pages, was 251 |
| protected-terms (scoped) | passed, 3,654 pages |
| manifest-invariants | holds, 203 x 18 |
| translation lib tests | 51 pass / 0 fail |
| menu-titles-fresh | up to date, 19 files |
| non-`---` content lines changed | 0 |

Incidentally a first real exercise of #1973: with the identical content repair but
the manifest left untagged, `manifest-invariants` now fails with 260 violations.
Before #1973 it passed green, so this repair could have landed with no provenance
trace at all.

Not done here, having checked and found nothing wrong: the 108 translated pages
using the markdown badge form `[![Edit](...)](...)` rather than the HTML
`<a><img/></a>` form. All 108 match their English source exactly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants